Controlled abstention neural networks for identifying skillful predictions for classification problems

The earth system is exceedingly complex and often chaotic in nature, making prediction incredibly challenging: we cannot expect to make perfect predictions all of the time. Instead, look for specific states that lead more predictable behavior than others, termed "forecasts opportunity." When these opportunities are not present, scientists need systems capable saying "I don't know." We introduce a novel loss function, "NotWrong loss", allows neural networks identify forecasts opportunity classification problems. NotWrong introduces an abstention class network confident samples abstain (say know") on less samples. designed user-defined fraction via PID controller. Unlike many machine learning methods used reject post-training, applied during training preferentially learn from show outperforms other existing functions multiple climate use cases. implementation proposed function straightforward most architectures as it only requires addition output layer modification function.